Pixel explanations for a deep neural network diabetic retinopathy classification model

Let's import some libraries required for the notebook to work:

In [1]:

Let's define some constants related with the model:

In [2]:

And now, let's load the test dataset:

In [3]:

Let's load the previously trained model:

In [4]:
=> loading checkpoint 'models/ret6_bn/model_best-QWKval0814.pth.tar'
=> loaded checkpoint 'models/ret6_bn/model_best-QWKval0814.pth.tar' (epoch 638)

We instantiate the previously defined model with the parameters of the pretrained one loaded before:

In [5]:
Out[5]:
model_explainable(
  (rf3): Sequential(
    (0): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf5): Sequential(
    (0): Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf9): Sequential(
    (0): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf13): Sequential(
    (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf21): Sequential(
    (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf29): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf45): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf61): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf93): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf125): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf189): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf253): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf381): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf509): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (rf637): Sequential(
    (0): Conv2d(64, 64, kernel_size=(2, 2), stride=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
  )
  (lc): Sequential(
    (0): Linear(in_features=64, out_features=5, bias=True)
  )
)

Now, we load a image from the dataset. This image is the one that will be analyzed:

In [6]:

Let's visualize the image:

In [7]:

Let's calculate with the model the predicted classification class. We also calculate the intermediate values of the activations of every layer

In [8]:
Image nr:999
File: ../input/640_messidor/3/20051021_59459_0100_PP.jpeg
Image name: 20051021_59459_0100_PP
Tag
Output before softmax: 
tensor([[-450.3686, -228.4829,  -53.1888,   37.3385,   73.2811]],
       grad_fn=<AddmmBackward>)
Predicted class: tensor(4)
Target: 
tensor([3])

Last layer feature activations

And now we plot the feature activations in the last layer of the model previous to the output layer:

In [9]:

We create a variable to sum the constants contributions of every layer mapped into the input space. Finally we will add this value to the values calculated for the input space

In [10]:

Score propagation through average pooling layer:

In [11]:
Max: 1.7733975648880005, Min: -5.512741565704346, Avg: -0.12144117057323456, Std:0.5413805246353149
torch.Size([5, 64, 4, 4])
Out[11]:
(tensor(1.7734), tensor(-5.5127), tensor(-0.1214), tensor(0.5414))
In [12]:
Out[12]:
tensor([[[[-1.2729e-01, -1.8573e-01, -8.1555e-03, -0.0000e+00],
          [-2.5151e-01, -2.6012e-01, -8.2957e-03, -0.0000e+00],
          [-6.8432e-02, -1.8465e-02, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -9.0093e-02, -3.9152e-02, -0.0000e+00]],

         [[ 0.0000e+00,  5.1282e-02,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  8.9081e-02,  2.0719e-01,  1.1845e-01]],

         [[-1.9736e-01, -2.7214e-01, -4.9350e-02, -0.0000e+00],
          [-4.0142e-01, -3.8694e-01, -8.7215e-02, -0.0000e+00],
          [-1.4519e-01, -5.1238e-02, -0.0000e+00, -0.0000e+00],
          [-6.7510e-03, -1.9274e-01, -2.2953e-01, -0.0000e+00]],

         ...,

         [[-9.8751e-02, -8.6059e-02, -5.8855e-02, -2.5383e-02],
          [-5.1446e-02, -6.8398e-02, -2.7565e-02, -6.6154e-02],
          [-2.2628e-01, -3.6083e-01, -1.9366e-01, -1.6756e-01],
          [-3.6341e-01, -4.1056e-01, -2.4784e-01, -1.7781e-01]],

         [[ 0.0000e+00,  1.7850e-02,  0.0000e+00,  0.0000e+00],
          [ 2.0058e-02,  0.0000e+00,  0.0000e+00,  9.1349e-03],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  6.9047e-03],
          [ 0.0000e+00,  1.1563e-02,  7.8865e-02,  2.4960e-02]],

         [[ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  3.0044e-02],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  9.7553e-02,  1.3223e-01,  0.0000e+00]]],


        [[[ 1.0932e-01,  1.5951e-01,  7.0042e-03,  0.0000e+00],
          [ 2.1600e-01,  2.2339e-01,  7.1246e-03,  0.0000e+00],
          [ 5.8772e-02,  1.5858e-02,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  7.7374e-02,  3.3625e-02,  0.0000e+00]],

         [[ 0.0000e+00,  3.3083e-02,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  5.7468e-02,  1.3366e-01,  7.6413e-02]],

         [[ 1.2925e-01,  1.7822e-01,  3.2318e-02,  0.0000e+00],
          [ 2.6287e-01,  2.5339e-01,  5.7114e-02,  0.0000e+00],
          [ 9.5078e-02,  3.3554e-02,  0.0000e+00,  0.0000e+00],
          [ 4.4210e-03,  1.2622e-01,  1.5031e-01,  0.0000e+00]],

         ...,

         [[-1.0416e-01, -9.0775e-02, -6.2080e-02, -2.6774e-02],
          [-5.4265e-02, -7.2146e-02, -2.9075e-02, -6.9779e-02],
          [-2.3868e-01, -3.8060e-01, -2.0428e-01, -1.7674e-01],
          [-3.8333e-01, -4.3306e-01, -2.6142e-01, -1.8755e-01]],

         [[ 0.0000e+00,  3.1602e-02,  0.0000e+00,  0.0000e+00],
          [ 3.5511e-02,  0.0000e+00,  0.0000e+00,  1.6173e-02],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  1.2224e-02],
          [ 0.0000e+00,  2.0471e-02,  1.3962e-01,  4.4189e-02]],

         [[ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  1.9339e-02],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  6.2793e-02,  8.5111e-02,  0.0000e+00]]],


        [[[ 5.0116e-02,  7.3120e-02,  3.2108e-03,  0.0000e+00],
          [ 9.9020e-02,  1.0241e-01,  3.2660e-03,  0.0000e+00],
          [ 2.6942e-02,  7.2695e-03,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  3.5470e-02,  1.5414e-02,  0.0000e+00]],

         [[ 0.0000e+00,  8.6414e-03,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  1.5011e-02,  3.4913e-02,  1.9959e-02]],

         [[ 7.2320e-02,  9.9723e-02,  1.8084e-02,  0.0000e+00],
          [ 1.4709e-01,  1.4179e-01,  3.1959e-02,  0.0000e+00],
          [ 5.3202e-02,  1.8775e-02,  0.0000e+00,  0.0000e+00],
          [ 2.4738e-03,  7.0628e-02,  8.4108e-02,  0.0000e+00]],

         ...,

         [[-1.0533e-01, -9.1790e-02, -6.2774e-02, -2.7073e-02],
          [-5.4871e-02, -7.2953e-02, -2.9400e-02, -7.0559e-02],
          [-2.4135e-01, -3.8486e-01, -2.0656e-01, -1.7872e-01],
          [-3.8761e-01, -4.3790e-01, -2.6434e-01, -1.8965e-01]],

         [[ 0.0000e+00,  2.1131e-02,  0.0000e+00,  0.0000e+00],
          [ 2.3744e-02,  0.0000e+00,  0.0000e+00,  1.0814e-02],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00,  8.1738e-03],
          [ 0.0000e+00,  1.3688e-02,  9.3360e-02,  2.9547e-02]],

         [[-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -1.4338e-02],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -4.6556e-02, -6.3103e-02, -0.0000e+00]]],


        [[[ 3.7343e-02,  5.4485e-02,  2.3925e-03,  0.0000e+00],
          [ 7.3783e-02,  7.6307e-02,  2.4336e-03,  0.0000e+00],
          [ 2.0075e-02,  5.4168e-03,  0.0000e+00,  0.0000e+00],
          [ 0.0000e+00,  2.6430e-02,  1.1486e-02,  0.0000e+00]],

         [[-0.0000e+00, -1.2684e-01, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -2.2034e-01, -5.1248e-01, -2.9297e-01]],

         [[ 2.2853e-02,  3.1512e-02,  5.7143e-03,  0.0000e+00],
          [ 4.6480e-02,  4.4804e-02,  1.0099e-02,  0.0000e+00],
          [ 1.6811e-02,  5.9329e-03,  0.0000e+00,  0.0000e+00],
          [ 7.8170e-04,  2.2318e-02,  2.6577e-02,  0.0000e+00]],

         ...,

         [[-3.8856e-02, -3.3862e-02, -2.3158e-02, -9.9875e-03],
          [-2.0243e-02, -2.6913e-02, -1.0846e-02, -2.6030e-02],
          [-8.9035e-02, -1.4198e-01, -7.6203e-02, -6.5931e-02],
          [-1.4299e-01, -1.6155e-01, -9.7519e-02, -6.9962e-02]],

         [[-0.0000e+00, -4.1176e-02, -0.0000e+00, -0.0000e+00],
          [-4.6269e-02, -0.0000e+00, -0.0000e+00, -2.1072e-02],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -1.5928e-02],
          [-0.0000e+00, -2.6673e-02, -1.8192e-01, -5.7576e-02]],

         [[-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -5.1285e-02],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -1.6652e-01, -2.2571e-01, -0.0000e+00]]],


        [[[-2.0569e-01, -3.0011e-01, -1.3178e-02, -0.0000e+00],
          [-4.0642e-01, -4.2032e-01, -1.3405e-02, -0.0000e+00],
          [-1.1058e-01, -2.9837e-02, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -1.4558e-01, -6.3266e-02, -0.0000e+00]],

         [[-0.0000e+00, -1.6025e-01, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -2.7836e-01, -6.4743e-01, -3.7012e-01]],

         [[-1.6418e-01, -2.2639e-01, -4.1053e-02, -0.0000e+00],
          [-3.3392e-01, -3.2188e-01, -7.2551e-02, -0.0000e+00],
          [-1.2078e-01, -4.2623e-02, -0.0000e+00, -0.0000e+00],
          [-5.6159e-03, -1.6034e-01, -1.9094e-01, -0.0000e+00]],

         ...,

         [[ 1.8998e-01,  1.6557e-01,  1.1323e-01,  4.8833e-02],
          [ 9.8974e-02,  1.3159e-01,  5.3031e-02,  1.2727e-01],
          [ 4.3533e-01,  6.9418e-01,  3.7258e-01,  3.2236e-01],
          [ 6.9915e-01,  7.8986e-01,  4.7681e-01,  3.4207e-01]],

         [[-0.0000e+00, -1.6179e-01, -0.0000e+00, -0.0000e+00],
          [-1.8180e-01, -0.0000e+00, -0.0000e+00, -8.2796e-02],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -6.2582e-02],
          [-0.0000e+00, -1.0480e-01, -7.1481e-01, -2.2623e-01]],

         [[-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -1.1010e-01],
          [-0.0000e+00, -0.0000e+00, -0.0000e+00, -0.0000e+00],
          [-0.0000e+00, -3.5749e-01, -4.8455e-01, -0.0000e+00]]]],
       grad_fn=<DivBackward0>)

Scores of the individual feature maps of the 15th layer

In [13]:

Aggregate feature maps scores of the 15th layer

In [14]:

Aggregated scores of 15th layer mapped into pixel space using a 2d-normal 2std=rf prior

In [15]:
tensor(-621.7795)

Score propagation through classification layer (2x2 convolution)

In [16]:
Max: 26.60015106201172, Min: -23.619178771972656, Avg: -0.12175504863262177, Std:1.2985899448394775
torch.Size([5, 64, 5, 5])
Out[16]:
(tensor(26.6002), tensor(-23.6192), tensor(-0.1218), tensor(1.2986))
In [17]:
torch.Size([5, 1, 4, 4])

In the previous map, every pixel corresponds to a receptive field. There is superposition between the activations showed in the above map. We now from bibliography that the receptive fields have a nearly gaussian detection capability. Considering a 2 stdev gaussian model over the receptive field, we can map the activations to input space. The result is shown below:

In [18]:
Max: 0.0011956972302868962, Min: -0.00044823819189332426, Avg: 0.00017200277943629771, Std:0.00041517912177369
torch.Size([5, 1, 640, 640])
tensor(352.2615)
tensor(352.2617)

Scores of the feature maps of the 14th layer

In [19]:

Aggregated feature map scores of the 14th layer

In [20]:

Aggregated feature map scores of the 14th layer mapped into input space

In [21]:
In [22]:
Max: 6.184974670410156, Min: -12.86451244354248, Avg: -0.02001095563173294, Std:0.4057272672653198
torch.Size([5, 64, 10, 10])
Max: 15.52629566192627, Min: -14.045604705810547, Avg: -0.010427805595099926, Std:0.3948817253112793
torch.Size([5, 64, 10, 10])
Out[22]:
(tensor(15.5263), tensor(-14.0456), tensor(-0.0104), tensor(0.3949))
In [23]:
torch.Size([5, 1, 10, 10])
In [24]:
Max: 0.0004063979140482843, Min: -0.001102820853702724, Avg: -0.00016293414228130132, Std:0.0003143970970995724
torch.Size([5, 1, 640, 640])
tensor(-333.6898)
tensor(-333.6891)
In [25]:
In [26]:
In [27]:
In [28]:
Max: 7.547794342041016, Min: -9.650070190429688, Avg: -0.02277785912156105, Std:0.4397107660770416
torch.Size([5, 64, 10, 10])
Out[28]:
(tensor(7.5478), tensor(-9.6501), tensor(-0.0228), tensor(0.4397))
In [29]:
torch.Size([5, 1, 10, 10])
In [30]:
Max: 0.0009819482220336795, Min: -0.0005125020979903638, Avg: 4.323295070207678e-05, Std:0.00022728461772203445
torch.Size([5, 1, 640, 640])
Out[30]:
tensor([[[[ 4.9132e-04,  4.9351e-04,  4.9570e-04,  ...,  9.6718e-05,
            9.6025e-05,  9.5334e-05],
          [ 4.9274e-04,  4.9494e-04,  4.9713e-04,  ...,  9.7393e-05,
            9.6693e-05,  9.5995e-05],
          [ 4.9415e-04,  4.9634e-04,  4.9853e-04,  ...,  9.8067e-05,
            9.7360e-05,  9.6655e-05],
          ...,
          [ 1.6990e-04,  1.6953e-04,  1.6916e-04,  ...,  3.2948e-04,
            3.2737e-04,  3.2527e-04],
          [ 1.6979e-04,  1.6943e-04,  1.6906e-04,  ...,  3.2799e-04,
            3.2590e-04,  3.2382e-04],
          [ 1.6968e-04,  1.6933e-04,  1.6897e-04,  ...,  3.2649e-04,
            3.2442e-04,  3.2235e-04]]],


        [[[ 2.2934e-04,  2.3049e-04,  2.3163e-04,  ...,  4.1443e-05,
            4.1286e-05,  4.1126e-05],
          [ 2.3004e-04,  2.3119e-04,  2.3234e-04,  ...,  4.1647e-05,
            4.1489e-05,  4.1328e-05],
          [ 2.3073e-04,  2.3188e-04,  2.3303e-04,  ...,  4.1849e-05,
            4.1690e-05,  4.1529e-05],
          ...,
          [ 5.8841e-05,  5.8417e-05,  5.7988e-05,  ...,  5.8342e-05,
            5.8324e-05,  5.8303e-05],
          [ 5.8938e-05,  5.8518e-05,  5.8094e-05,  ...,  5.8248e-05,
            5.8231e-05,  5.8210e-05],
          [ 5.9034e-05,  5.8619e-05,  5.8199e-05,  ...,  5.8153e-05,
            5.8136e-05,  5.8116e-05]]],


        [[[ 3.3029e-05,  3.3153e-05,  3.3274e-05,  ...,  1.4375e-05,
            1.4480e-05,  1.4581e-05],
          [ 3.3041e-05,  3.3163e-05,  3.3282e-05,  ...,  1.4395e-05,
            1.4501e-05,  1.4604e-05],
          [ 3.3047e-05,  3.3167e-05,  3.3285e-05,  ...,  1.4415e-05,
            1.4522e-05,  1.4626e-05],
          ...,
          [-6.7970e-06, -7.2330e-06, -7.6747e-06,  ..., -1.0014e-04,
           -9.8959e-05, -9.7783e-05],
          [-6.5319e-06, -6.9639e-06, -7.4015e-06,  ..., -9.9411e-05,
           -9.8238e-05, -9.7071e-05],
          [-6.2691e-06, -6.6971e-06, -7.1306e-06,  ..., -9.8679e-05,
           -9.7514e-05, -9.6356e-05]]],


        [[[-4.6899e-05, -4.7076e-05, -4.7252e-05,  ..., -1.3470e-05,
           -1.3431e-05, -1.3391e-05],
          [-4.6985e-05, -4.7162e-05, -4.7336e-05,  ..., -1.3542e-05,
           -1.3503e-05, -1.3463e-05],
          [-4.7068e-05, -4.7243e-05, -4.7416e-05,  ..., -1.3614e-05,
           -1.3575e-05, -1.3535e-05],
          ...,
          [-4.0939e-06, -3.8866e-06, -3.6787e-06,  ...,  4.2183e-06,
            4.0857e-06,  3.9539e-06],
          [-4.2465e-06, -4.0418e-06, -3.8365e-06,  ...,  4.1279e-06,
            3.9961e-06,  3.8652e-06],
          [-4.3985e-06, -4.1964e-06, -3.9936e-06,  ...,  4.0382e-06,
            3.9074e-06,  3.7774e-06]]],


        [[[-1.3690e-04, -1.3761e-04, -1.3832e-04,  ..., -4.6318e-05,
           -4.6351e-05, -4.6381e-05],
          [-1.3743e-04, -1.3814e-04, -1.3885e-04,  ..., -4.6529e-05,
           -4.6563e-05, -4.6594e-05],
          [-1.3795e-04, -1.3867e-04, -1.3938e-04,  ..., -4.6741e-05,
           -4.6775e-05, -4.6807e-05],
          ...,
          [-6.8632e-05, -6.8441e-05, -6.8246e-05,  ...,  3.2890e-05,
            3.1959e-05,  3.1036e-05],
          [-6.8678e-05, -6.8491e-05, -6.8300e-05,  ...,  3.2399e-05,
            3.1475e-05,  3.0560e-05],
          [-6.8723e-05, -6.8539e-05, -6.8351e-05,  ...,  3.1905e-05,
            3.0990e-05,  3.0081e-05]]]])
In [31]:
In [32]:
In [33]:
Max: 5.6382927894592285, Min: -17.000303268432617, Avg: -0.0064177303574979305, Std:0.2180030345916748
torch.Size([5, 64, 20, 20])
Out[33]:
(tensor(5.6383), tensor(-17.0003), tensor(-0.0064), tensor(0.2180))
In [34]:
torch.Size([5, 1, 20, 20])
In [35]:
In [36]:
Max: 0.0012812407221645117, Min: -0.0011766179231926799, Avg: 4.5204022171674296e-05, Std:0.00033224321668967605
torch.Size([5, 1, 640, 640])
Out[36]:
tensor([[[[ 8.6904e-04,  8.7407e-04,  8.7902e-04,  ...,  4.5333e-04,
            4.5012e-04,  4.4685e-04],
          [ 8.7310e-04,  8.7814e-04,  8.8311e-04,  ...,  4.5726e-04,
            4.5401e-04,  4.5071e-04],
          [ 8.7709e-04,  8.8214e-04,  8.8711e-04,  ...,  4.6115e-04,
            4.5787e-04,  4.5454e-04],
          ...,
          [ 4.5815e-04,  4.6005e-04,  4.6190e-04,  ...,  9.3365e-04,
            9.2879e-04,  9.2381e-04],
          [ 4.5562e-04,  4.5752e-04,  4.5936e-04,  ...,  9.2779e-04,
            9.2296e-04,  9.1800e-04],
          [ 4.5306e-04,  4.5495e-04,  4.5679e-04,  ...,  9.2183e-04,
            9.1703e-04,  9.1210e-04]]],


        [[[ 3.6144e-04,  3.6312e-04,  3.6476e-04,  ...,  1.8827e-04,
            1.8704e-04,  1.8578e-04],
          [ 3.6317e-04,  3.6485e-04,  3.6649e-04,  ...,  1.8976e-04,
            1.8852e-04,  1.8725e-04],
          [ 3.6486e-04,  3.6655e-04,  3.6819e-04,  ...,  1.9124e-04,
            1.8998e-04,  1.8870e-04],
          ...,
          [ 2.7692e-04,  2.7846e-04,  2.7996e-04,  ...,  2.9322e-04,
            2.9199e-04,  2.9071e-04],
          [ 2.7510e-04,  2.7662e-04,  2.7812e-04,  ...,  2.9182e-04,
            2.9059e-04,  2.8931e-04],
          [ 2.7324e-04,  2.7476e-04,  2.7625e-04,  ...,  2.9037e-04,
            2.8914e-04,  2.8786e-04]]],


        [[[ 1.0356e-05,  9.9515e-06,  9.5368e-06,  ...,  6.5485e-05,
            6.5172e-05,  6.4850e-05],
          [ 1.0218e-05,  9.8073e-06,  9.3870e-06,  ...,  6.5916e-05,
            6.5601e-05,  6.5276e-05],
          [ 1.0075e-05,  9.6598e-06,  9.2340e-06,  ...,  6.6341e-05,
            6.6023e-05,  6.5695e-05],
          ...,
          [ 6.7669e-05,  6.8133e-05,  6.8588e-05,  ..., -3.7212e-05,
           -3.6797e-05, -3.6389e-05],
          [ 6.7125e-05,  6.7584e-05,  6.8034e-05,  ..., -3.6677e-05,
           -3.6269e-05, -3.5869e-05],
          [ 6.6574e-05,  6.7028e-05,  6.7473e-05,  ..., -3.6154e-05,
           -3.5754e-05, -3.5361e-05]]],


        [[[-6.9852e-05, -7.0217e-05, -7.0576e-05,  ..., -5.6760e-05,
           -5.6397e-05, -5.6028e-05],
          [-7.0132e-05, -7.0497e-05, -7.0855e-05,  ..., -5.7241e-05,
           -5.6875e-05, -5.6502e-05],
          [-7.0406e-05, -7.0770e-05, -7.1128e-05,  ..., -5.7718e-05,
           -5.7349e-05, -5.6973e-05],
          ...,
          [-3.3862e-05, -3.3866e-05, -3.3863e-05,  ..., -5.8156e-05,
           -5.7912e-05, -5.7655e-05],
          [-3.3686e-05, -3.3691e-05, -3.3689e-05,  ..., -5.7741e-05,
           -5.7499e-05, -5.7244e-05],
          [-3.3507e-05, -3.3513e-05, -3.3513e-05,  ..., -5.7318e-05,
           -5.7079e-05, -5.6826e-05]]],


        [[[-2.0484e-04, -2.0557e-04, -2.0628e-04,  ..., -1.5616e-04,
           -1.5535e-04, -1.5451e-04],
          [-2.0576e-04, -2.0649e-04, -2.0719e-04,  ..., -1.5741e-04,
           -1.5659e-04, -1.5575e-04],
          [-2.0665e-04, -2.0738e-04, -2.0809e-04,  ..., -1.5866e-04,
           -1.5783e-04, -1.5698e-04],
          ...,
          [-2.0460e-04, -2.0577e-04, -2.0693e-04,  ..., -1.1352e-04,
           -1.1321e-04, -1.1288e-04],
          [-2.0336e-04, -2.0452e-04, -2.0568e-04,  ..., -1.1334e-04,
           -1.1303e-04, -1.1269e-04],
          [-2.0210e-04, -2.0327e-04, -2.0441e-04,  ..., -1.1314e-04,
           -1.1282e-04, -1.1248e-04]]]])
In [37]:
In [38]:
In [39]:
In [40]:
Max: 6.0583577156066895, Min: -19.04521369934082, Avg: -0.005227937828749418, Std:0.2634696960449219
torch.Size([5, 64, 20, 20])
Out[40]:
(tensor(6.0584), tensor(-19.0452), tensor(-0.0052), tensor(0.2635))
In [41]:
torch.Size([5, 1, 20, 20])
In [42]:
Max: 0.003454436082392931, Min: -0.0025498869363218546, Avg: -7.43619239074178e-05, Std:0.0007212458876892924
torch.Size([5, 1, 640, 640])
Out[42]:
tensor([[[[ 4.5806e-04,  4.5703e-04,  4.5599e-04,  ...,  4.0530e-04,
            4.0306e-04,  4.0076e-04],
          [ 4.5470e-04,  4.5357e-04,  4.5243e-04,  ...,  4.0603e-04,
            4.0381e-04,  4.0154e-04],
          [ 4.5127e-04,  4.5003e-04,  4.4880e-04,  ...,  4.0664e-04,
            4.0444e-04,  4.0219e-04],
          ...,
          [ 5.9320e-05,  5.7315e-05,  5.5380e-05,  ...,  1.0540e-03,
            1.0472e-03,  1.0402e-03],
          [ 6.3106e-05,  6.1156e-05,  5.9275e-05,  ...,  1.0496e-03,
            1.0428e-03,  1.0358e-03],
          [ 6.6896e-05,  6.5002e-05,  6.3174e-05,  ...,  1.0449e-03,
            1.0381e-03,  1.0311e-03]]],


        [[[ 9.3733e-05,  9.1558e-05,  8.9388e-05,  ...,  1.3120e-04,
            1.3089e-04,  1.3056e-04],
          [ 9.1499e-05,  8.9271e-05,  8.7049e-05,  ...,  1.3098e-04,
            1.3069e-04,  1.3037e-04],
          [ 8.9266e-05,  8.6986e-05,  8.4713e-05,  ...,  1.3071e-04,
            1.3044e-04,  1.3014e-04],
          ...,
          [ 4.6280e-05,  4.5055e-05,  4.3844e-05,  ...,  2.7407e-04,
            2.7240e-04,  2.7070e-04],
          [ 4.8321e-05,  4.7131e-05,  4.5954e-05,  ...,  2.7312e-04,
            2.7145e-04,  2.6976e-04],
          [ 5.0347e-05,  4.9191e-05,  4.8048e-05,  ...,  2.7213e-04,
            2.7047e-04,  2.6879e-04]]],


        [[[-4.0988e-05, -4.1864e-05, -4.2734e-05,  ...,  3.3615e-05,
            3.3785e-05,  3.3951e-05],
          [-4.1545e-05, -4.2428e-05, -4.3305e-05,  ...,  3.3327e-05,
            3.3507e-05,  3.3682e-05],
          [-4.2089e-05, -4.2979e-05, -4.3862e-05,  ...,  3.3028e-05,
            3.3218e-05,  3.3402e-05],
          ...,
          [ 2.1435e-05,  2.1125e-05,  2.0809e-05,  ..., -1.0350e-04,
           -1.0267e-04, -1.0180e-04],
          [ 2.1874e-05,  2.1574e-05,  2.1267e-05,  ..., -1.0266e-04,
           -1.0183e-04, -1.0097e-04],
          [ 2.2301e-05,  2.2011e-05,  2.1713e-05,  ..., -1.0179e-04,
           -1.0096e-04, -1.0011e-04]]],


        [[[-6.2871e-05, -6.3161e-05, -6.3449e-05,  ..., -5.5676e-05,
           -5.5325e-05, -5.4968e-05],
          [-6.2664e-05, -6.2946e-05, -6.3226e-05,  ..., -5.5856e-05,
           -5.5506e-05, -5.5151e-05],
          [-6.2437e-05, -6.2711e-05, -6.2983e-05,  ..., -5.6021e-05,
           -5.5673e-05, -5.5320e-05],
          ...,
          [ 1.8100e-05,  1.8508e-05,  1.8902e-05,  ..., -5.1195e-05,
           -5.1053e-05, -5.0898e-05],
          [ 1.7360e-05,  1.7759e-05,  1.8145e-05,  ..., -5.1027e-05,
           -5.0886e-05, -5.0732e-05],
          [ 1.6621e-05,  1.7011e-05,  1.7389e-05,  ..., -5.0848e-05,
           -5.0707e-05, -5.0553e-05]]],


        [[[-2.9461e-05, -2.7750e-05, -2.6044e-05,  ..., -1.0442e-04,
           -1.0438e-04, -1.0433e-04],
          [-2.8360e-05, -2.6623e-05, -2.4891e-05,  ..., -1.0439e-04,
           -1.0437e-04, -1.0433e-04],
          [-2.7282e-05, -2.5519e-05, -2.3762e-05,  ..., -1.0435e-04,
           -1.0434e-04, -1.0432e-04],
          ...,
          [-7.8940e-05, -7.8297e-05, -7.7652e-05,  ...,  1.6497e-05,
            1.5564e-05,  1.4611e-05],
          [-7.9586e-05, -7.8962e-05, -7.8337e-05,  ...,  1.5559e-05,
            1.4637e-05,  1.3698e-05],
          [-8.0233e-05, -7.9629e-05, -7.9023e-05,  ...,  1.4593e-05,
            1.3684e-05,  1.2757e-05]]]])
In [43]:
In [44]:
In [45]:
In [46]:
Max: 8.269240379333496, Min: -9.289427757263184, Avg: -0.0006222487427294254, Std:0.08974684774875641
torch.Size([5, 64, 40, 40])
Out[46]:
(tensor(8.2692), tensor(-9.2894), tensor(-0.0006), tensor(0.0897))
In [47]:
torch.Size([5, 1, 40, 40])
In [48]:
Max: 0.011518329381942749, Min: -0.015644419938325882, Avg: -0.0001711840450298041, Std:0.0017444925615563989
torch.Size([5, 1, 640, 640])
Out[48]:
tensor([[[[ 1.2569e-03,  1.2682e-03,  1.2788e-03,  ...,  4.6026e-04,
            4.5607e-04,  4.5190e-04],
          [ 1.2685e-03,  1.2799e-03,  1.2907e-03,  ...,  4.6138e-04,
            4.5720e-04,  4.5304e-04],
          [ 1.2796e-03,  1.2911e-03,  1.3021e-03,  ...,  4.6233e-04,
            4.5817e-04,  4.5402e-04],
          ...,
          [ 2.8333e-04,  2.8599e-04,  2.8859e-04,  ...,  1.4266e-03,
            1.4126e-03,  1.3982e-03],
          [ 2.7643e-04,  2.7898e-04,  2.8149e-04,  ...,  1.4204e-03,
            1.4064e-03,  1.3922e-03],
          [ 2.6983e-04,  2.7229e-04,  2.7472e-04,  ...,  1.4135e-03,
            1.3996e-03,  1.3855e-03]]],


        [[[ 2.6720e-04,  2.6784e-04,  2.6837e-04,  ...,  1.5603e-04,
            1.5514e-04,  1.5425e-04],
          [ 2.6888e-04,  2.6954e-04,  2.7008e-04,  ...,  1.5572e-04,
            1.5485e-04,  1.5398e-04],
          [ 2.7053e-04,  2.7119e-04,  2.7174e-04,  ...,  1.5534e-04,
            1.5450e-04,  1.5365e-04],
          ...,
          [ 6.1991e-05,  6.0282e-05,  5.8532e-05,  ...,  2.8361e-04,
            2.8100e-04,  2.7841e-04],
          [ 6.0888e-05,  5.9196e-05,  5.7466e-05,  ...,  2.8522e-04,
            2.8258e-04,  2.7997e-04],
          [ 5.9932e-05,  5.8260e-05,  5.6552e-05,  ...,  2.8664e-04,
            2.8398e-04,  2.8136e-04]]],


        [[[-1.6465e-04, -1.6752e-04, -1.7031e-04,  ...,  3.1804e-05,
            3.2108e-05,  3.2414e-05],
          [-1.6709e-04, -1.7000e-04, -1.7283e-04,  ...,  3.1156e-05,
            3.1483e-05,  3.1812e-05],
          [-1.6945e-04, -1.7240e-04, -1.7526e-04,  ...,  3.0485e-05,
            3.0836e-05,  3.1187e-05],
          ...,
          [ 1.4046e-05,  1.2898e-05,  1.1714e-05,  ..., -1.7474e-04,
           -1.7322e-04, -1.7158e-04],
          [ 1.3720e-05,  1.2586e-05,  1.1416e-05,  ..., -1.7147e-04,
           -1.6999e-04, -1.6841e-04],
          [ 1.3437e-05,  1.2317e-05,  1.1162e-05,  ..., -1.6820e-04,
           -1.6676e-04, -1.6523e-04]]],


        [[[-1.2978e-04, -1.3082e-04, -1.3180e-04,  ..., -5.5437e-05,
           -5.4974e-05, -5.4518e-05],
          [-1.3093e-04, -1.3198e-04, -1.3297e-04,  ..., -5.5390e-05,
           -5.4932e-05, -5.4481e-05],
          [-1.3203e-04, -1.3309e-04, -1.3409e-04,  ..., -5.5319e-05,
           -5.4866e-05, -5.4421e-05],
          ...,
          [ 2.1476e-05,  2.1941e-05,  2.2397e-05,  ..., -7.5883e-05,
           -7.5893e-05, -7.5855e-05],
          [ 2.1978e-05,  2.2452e-05,  2.2914e-05,  ..., -7.6258e-05,
           -7.6247e-05, -7.6188e-05],
          [ 2.2402e-05,  2.2882e-05,  2.3350e-05,  ..., -7.6560e-05,
           -7.6529e-05, -7.6451e-05]]],


        [[[-7.8842e-05, -7.8152e-05, -7.7445e-05,  ..., -1.4722e-04,
           -1.4641e-04, -1.4558e-04],
          [-7.8816e-05, -7.8117e-05, -7.7400e-05,  ..., -1.4740e-04,
           -1.4660e-04, -1.4579e-04],
          [-7.8796e-05, -7.8088e-05, -7.7362e-05,  ..., -1.4751e-04,
           -1.4673e-04, -1.4594e-04],
          ...,
          [-7.5195e-05, -7.3759e-05, -7.2281e-05,  ...,  9.5740e-05,
            9.4117e-05,  9.2376e-05],
          [-7.4245e-05, -7.2830e-05, -7.1377e-05,  ...,  9.1342e-05,
            8.9788e-05,  8.8120e-05],
          [-7.3398e-05, -7.2007e-05, -7.0578e-05,  ...,  8.6976e-05,
            8.5491e-05,  8.3894e-05]]]])
In [49]:
In [50]:
In [51]:
In [52]:
Max: 4.628687858581543, Min: -9.527755737304688, Avg: 0.00017323445354122669, Std:0.10171706974506378
torch.Size([5, 64, 40, 40])
Out[52]:
(tensor(4.6287), tensor(-9.5278), tensor(0.0002), tensor(0.1017))
In [53]:
torch.Size([5, 1, 40, 40])
In [54]:
Max: 0.010726242326200008, Min: -0.012327617034316063, Avg: -0.00019887101370841265, Std:0.001408573123626411
torch.Size([5, 1, 640, 640])
Out[54]:
tensor([[[[ 6.6911e-04,  6.6268e-04,  6.5611e-04,  ...,  4.6319e-04,
            4.5824e-04,  4.5335e-04],
          [ 6.6619e-04,  6.5949e-04,  6.5266e-04,  ...,  4.6381e-04,
            4.5888e-04,  4.5401e-04],
          [ 6.6350e-04,  6.5656e-04,  6.4948e-04,  ...,  4.6421e-04,
            4.5931e-04,  4.5446e-04],
          ...,
          [-5.5769e-05, -6.1614e-05, -6.7078e-05,  ...,  1.2887e-03,
            1.2807e-03,  1.2724e-03],
          [-5.2952e-05, -5.8642e-05, -6.3956e-05,  ...,  1.2886e-03,
            1.2805e-03,  1.2721e-03],
          [-4.9622e-05, -5.5149e-05, -6.0306e-05,  ...,  1.2878e-03,
            1.2796e-03,  1.2711e-03]]],


        [[[ 8.0166e-05,  7.5485e-05,  7.0840e-05,  ...,  1.6786e-04,
            1.6611e-04,  1.6439e-04],
          [ 7.7608e-05,  7.2831e-05,  6.8093e-05,  ...,  1.6801e-04,
            1.6626e-04,  1.6453e-04],
          [ 7.5246e-05,  7.0382e-05,  6.5558e-05,  ...,  1.6808e-04,
            1.6633e-04,  1.6460e-04],
          ...,
          [-9.5517e-05, -1.0037e-04, -1.0501e-04,  ...,  3.1285e-04,
            3.0989e-04,  3.0689e-04],
          [-9.2509e-05, -9.7293e-05, -1.0187e-04,  ...,  3.1235e-04,
            3.0945e-04,  3.0650e-04],
          [-8.9233e-05, -9.3940e-05, -9.8448e-05,  ...,  3.1174e-04,
            3.0888e-04,  3.0599e-04]]],


        [[[-1.0109e-04, -1.0195e-04, -1.0278e-04,  ...,  4.9611e-05,
            4.8920e-05,  4.8247e-05],
          [-1.0180e-04, -1.0265e-04, -1.0346e-04,  ...,  4.9876e-05,
            4.9160e-05,  4.8463e-05],
          [-1.0248e-04, -1.0332e-04, -1.0411e-04,  ...,  5.0136e-05,
            4.9394e-05,  4.8672e-05],
          ...,
          [-4.2127e-05, -4.4318e-05, -4.6455e-05,  ..., -1.1626e-04,
           -1.1618e-04, -1.1606e-04],
          [-4.0783e-05, -4.2943e-05, -4.5051e-05,  ..., -1.1677e-04,
           -1.1661e-04, -1.1642e-04],
          [-3.9369e-05, -4.1495e-05, -4.3571e-05,  ..., -1.1716e-04,
           -1.1693e-04, -1.1667e-04]]],


        [[[-8.8858e-05, -8.8652e-05, -8.8415e-05,  ..., -6.3128e-05,
           -6.2323e-05, -6.1526e-05],
          [-8.8837e-05, -8.8608e-05, -8.8348e-05,  ..., -6.3395e-05,
           -6.2583e-05, -6.1780e-05],
          [-8.8808e-05, -8.8557e-05, -8.8276e-05,  ..., -6.3638e-05,
           -6.2820e-05, -6.2011e-05],
          ...,
          [ 5.9079e-05,  6.0324e-05,  6.1496e-05,  ..., -9.0611e-05,
           -8.9754e-05, -8.8879e-05],
          [ 5.8515e-05,  5.9744e-05,  6.0902e-05,  ..., -9.0436e-05,
           -8.9592e-05, -8.8731e-05],
          [ 5.7849e-05,  5.9061e-05,  6.0203e-05,  ..., -9.0190e-05,
           -8.9361e-05, -8.8514e-05]]],


        [[[-2.3012e-05, -2.0680e-05, -1.8367e-05,  ..., -1.2507e-04,
           -1.2440e-04, -1.2375e-04],
          [-2.2025e-05, -1.9663e-05, -1.7319e-05,  ..., -1.2507e-04,
           -1.2440e-04, -1.2375e-04],
          [-2.1139e-05, -1.8748e-05, -1.6377e-05,  ..., -1.2506e-04,
           -1.2438e-04, -1.2373e-04],
          ...,
          [-1.8765e-05, -1.6684e-05, -1.4706e-05,  ...,  4.2914e-05,
            4.2365e-05,  4.1770e-05],
          [-1.9053e-05, -1.6991e-05, -1.5030e-05,  ...,  4.3287e-05,
            4.2654e-05,  4.1977e-05],
          [-1.9505e-05, -1.7466e-05, -1.5526e-05,  ...,  4.3493e-05,
            4.2785e-05,  4.2032e-05]]]])
In [55]:
In [56]:
In [57]:
In [58]:
Max: 2.695253849029541, Min: -5.955574989318848, Avg: 0.00025274729705415666, Std:0.037722814828157425
torch.Size([5, 64, 80, 80])
Out[58]:
(tensor(2.6953), tensor(-5.9556), tensor(0.0003), tensor(0.0377))
In [59]:
torch.Size([5, 1, 80, 80])
In [60]:
Max: 0.008690044283866882, Min: -0.02500220201909542, Avg: -0.00020943833806086332, Std:0.0017538812244310975
torch.Size([5, 1, 640, 640])
Out[60]:
tensor([[[[ 8.3334e-04,  8.3230e-04,  8.3078e-04,  ...,  5.0885e-04,
            5.0342e-04,  4.9786e-04],
          [ 8.3474e-04,  8.3356e-04,  8.3189e-04,  ...,  5.1087e-04,
            5.0549e-04,  4.9997e-04],
          [ 8.3593e-04,  8.3461e-04,  8.3280e-04,  ...,  5.1255e-04,
            5.0723e-04,  5.0176e-04],
          ...,
          [ 5.9232e-04,  6.1947e-04,  6.4610e-04,  ...,  1.2842e-03,
            1.2717e-03,  1.2593e-03],
          [ 5.7181e-04,  5.9795e-04,  6.2361e-04,  ...,  1.2797e-03,
            1.2674e-03,  1.2553e-03],
          [ 5.5048e-04,  5.7555e-04,  6.0016e-04,  ...,  1.2749e-03,
            1.2629e-03,  1.2511e-03]]],


        [[[ 9.5887e-05,  8.9252e-05,  8.2449e-05,  ...,  1.9455e-04,
            1.9211e-04,  1.8961e-04],
          [ 9.1698e-05,  8.4779e-05,  7.7688e-05,  ...,  1.9584e-04,
            1.9337e-04,  1.9085e-04],
          [ 8.7556e-05,  8.0361e-05,  7.2991e-05,  ...,  1.9699e-04,
            1.9451e-04,  1.9197e-04],
          ...,
          [ 2.4485e-04,  2.5569e-04,  2.6617e-04,  ...,  3.8290e-04,
            3.7362e-04,  3.6449e-04],
          [ 2.3635e-04,  2.4678e-04,  2.5686e-04,  ...,  3.7798e-04,
            3.6910e-04,  3.6036e-04],
          [ 2.2739e-04,  2.3737e-04,  2.4702e-04,  ...,  3.7298e-04,
            3.6449e-04,  3.5615e-04]]],


        [[[-1.4515e-04, -1.4863e-04, -1.5205e-04,  ...,  6.3086e-05,
            6.2183e-05,  6.1250e-05],
          [-1.4832e-04, -1.5194e-04, -1.5548e-04,  ...,  6.4117e-05,
            6.3183e-05,  6.2215e-05],
          [-1.5142e-04, -1.5515e-04, -1.5881e-04,  ...,  6.5136e-05,
            6.4170e-05,  6.3168e-05],
          ...,
          [ 5.8150e-05,  5.9933e-05,  6.1548e-05,  ..., -5.2575e-05,
           -5.6413e-05, -6.0206e-05],
          [ 5.6593e-05,  5.8329e-05,  5.9904e-05,  ..., -5.5992e-05,
           -5.9566e-05, -6.3096e-05],
          [ 5.4858e-05,  5.6534e-05,  5.8057e-05,  ..., -5.9405e-05,
           -6.2714e-05, -6.5981e-05]]],


        [[[-8.5884e-05, -8.5848e-05, -8.5826e-05,  ..., -7.1452e-05,
           -7.0382e-05, -6.9303e-05],
          [-8.5613e-05, -8.5561e-05, -8.5525e-05,  ..., -7.2103e-05,
           -7.1016e-05, -6.9919e-05],
          [-8.5338e-05, -8.5272e-05, -8.5223e-05,  ..., -7.2715e-05,
           -7.1612e-05, -7.0499e-05],
          ...,
          [-4.7273e-05, -5.1439e-05, -5.5532e-05,  ..., -1.0027e-04,
           -9.8494e-05, -9.6742e-05],
          [-4.3960e-05, -4.7950e-05, -5.1871e-05,  ..., -9.9118e-05,
           -9.7428e-05, -9.5756e-05],
          [-4.0534e-05, -4.4338e-05, -4.8077e-05,  ..., -9.7954e-05,
           -9.6346e-05, -9.4752e-05]]],


        [[[-3.3231e-05, -2.9620e-05, -2.5886e-05,  ..., -1.5302e-04,
           -1.5211e-04, -1.5110e-04],
          [-3.1289e-05, -2.7521e-05, -2.3625e-05,  ..., -1.5422e-04,
           -1.5332e-04, -1.5231e-04],
          [-2.9342e-05, -2.5418e-05, -2.1364e-05,  ..., -1.5537e-04,
           -1.5447e-04, -1.5347e-04],
          ...,
          [-2.4279e-04, -2.5153e-04, -2.6005e-04,  ..., -7.7369e-05,
           -7.0384e-05, -6.3491e-05],
          [-2.3497e-04, -2.4335e-04, -2.5152e-04,  ..., -7.1052e-05,
           -6.4515e-05, -5.8068e-05],
          [-2.2687e-04, -2.3487e-04, -2.4267e-04,  ..., -6.4760e-05,
           -5.8672e-05, -5.2673e-05]]]])
In [61]:
In [62]:
In [63]:
In [64]:
Max: 3.2925291061401367, Min: -5.844082832336426, Avg: 0.00010534186731092632, Std:0.041589122265577316
torch.Size([5, 64, 80, 80])
Out[64]:
(tensor(3.2925), tensor(-5.8441), tensor(0.0001), tensor(0.0416))
In [65]:
torch.Size([5, 1, 80, 80])
In [66]:
Max: 0.02499956265091896, Min: -0.01767553947865963, Avg: 0.00014740544429514557, Std:0.001804392784833908
torch.Size([5, 1, 640, 640])
Out[66]:
tensor([[[[ 7.9797e-04,  8.0185e-04,  8.0623e-04,  ...,  4.2182e-04,
            4.1368e-04,  4.0675e-04],
          [ 7.9220e-04,  7.9577e-04,  7.9987e-04,  ...,  4.2344e-04,
            4.1470e-04,  4.0726e-04],
          [ 7.8539e-04,  7.8854e-04,  7.9224e-04,  ...,  4.2571e-04,
            4.1632e-04,  4.0832e-04],
          ...,
          [ 6.2862e-04,  6.5380e-04,  6.7752e-04,  ...,  1.6016e-03,
            1.5755e-03,  1.5482e-03],
          [ 6.0845e-04,  6.3305e-04,  6.5628e-04,  ...,  1.5888e-03,
            1.5632e-03,  1.5364e-03],
          [ 5.8694e-04,  6.1083e-04,  6.3347e-04,  ...,  1.5734e-03,
            1.5484e-03,  1.5223e-03]]],


        [[[ 1.6470e-04,  1.6817e-04,  1.7196e-04,  ...,  1.3988e-04,
            1.3761e-04,  1.3585e-04],
          [ 1.6224e-04,  1.6589e-04,  1.6989e-04,  ...,  1.3953e-04,
            1.3707e-04,  1.3516e-04],
          [ 1.5917e-04,  1.6293e-04,  1.6708e-04,  ...,  1.3949e-04,
            1.3681e-04,  1.3472e-04],
          ...,
          [ 7.4206e-05,  7.5639e-05,  7.7456e-05,  ...,  3.6184e-04,
            3.5541e-04,  3.4885e-04],
          [ 7.2915e-05,  7.4287e-05,  7.6026e-05,  ...,  3.6048e-04,
            3.5420e-04,  3.4778e-04],
          [ 7.1894e-05,  7.3219e-05,  7.4891e-05,  ...,  3.5862e-04,
            3.5250e-04,  3.4623e-04]]],


        [[[-9.8458e-05, -9.7834e-05, -9.7138e-05,  ...,  2.6423e-05,
            2.5657e-05,  2.5234e-05],
          [-9.9062e-05, -9.8288e-05, -9.7432e-05,  ...,  2.5748e-05,
            2.4885e-05,  2.4388e-05],
          [-9.9751e-05, -9.8832e-05, -9.7820e-05,  ...,  2.5261e-05,
            2.4289e-05,  2.3706e-05],
          ...,
          [-2.0957e-05, -2.2469e-05, -2.3604e-05,  ..., -1.0093e-04,
           -1.0169e-04, -1.0240e-04],
          [-1.9867e-05, -2.1394e-05, -2.2566e-05,  ..., -1.0179e-04,
           -1.0241e-04, -1.0300e-04],
          [-1.8496e-05, -2.0019e-05, -2.1210e-05,  ..., -1.0254e-04,
           -1.0304e-04, -1.0351e-04]]],


        [[[-8.8710e-05, -8.9200e-05, -8.9740e-05,  ..., -5.5306e-05,
           -5.4196e-05, -5.3256e-05],
          [-8.8166e-05, -8.8608e-05, -8.9102e-05,  ..., -5.5434e-05,
           -5.4252e-05, -5.3253e-05],
          [-8.7533e-05, -8.7917e-05, -8.8353e-05,  ..., -5.5643e-05,
           -5.4384e-05, -5.3320e-05],
          ...,
          [-1.9961e-05, -2.2118e-05, -2.4232e-05,  ..., -1.1135e-04,
           -1.0855e-04, -1.0577e-04],
          [-1.8108e-05, -2.0208e-05, -2.2271e-05,  ..., -1.1041e-04,
           -1.0772e-04, -1.0506e-04],
          [-1.6200e-05, -1.8234e-05, -2.0237e-05,  ..., -1.0926e-04,
           -1.0670e-04, -1.0415e-04]]],


        [[[-6.4641e-05, -6.5535e-05, -6.6517e-05,  ..., -1.1585e-04,
           -1.1445e-04, -1.1343e-04],
          [-6.3605e-05, -6.4611e-05, -6.5726e-05,  ..., -1.1622e-04,
           -1.1464e-04, -1.1346e-04],
          [-6.2296e-05, -6.3397e-05, -6.4629e-05,  ..., -1.1688e-04,
           -1.1508e-04, -1.1373e-04],
          ...,
          [-1.5781e-04, -1.6205e-04, -1.6646e-04,  ..., -3.6881e-05,
           -3.2856e-05, -2.8821e-05],
          [-1.5347e-04, -1.5751e-04, -1.6171e-04,  ..., -3.3460e-05,
           -2.9758e-05, -2.6033e-05],
          [-1.4923e-04, -1.5307e-04, -1.5706e-04,  ..., -2.9972e-05,
           -2.6583e-05, -2.3164e-05]]]])
In [67]:
In [68]:
In [69]:
In [70]:
In [71]:
Max: 3.741147994995117, Min: -5.037511348724365, Avg: 1.6697027604095638e-05, Std:0.017496244981884956
torch.Size([5, 64, 160, 160])
Out[71]:
(tensor(3.7411), tensor(-5.0375), tensor(1.6697e-05), tensor(0.0175))
In [72]:
torch.Size([5, 1, 160, 160])
In [73]:
Max: 0.02855204977095127, Min: -0.10004856437444687, Avg: 3.855421891785227e-05, Std:0.0024134055711328983
torch.Size([5, 1, 640, 640])
Out[73]:
tensor([[[[ 5.7735e-04,  5.8094e-04,  5.9380e-04,  ...,  2.1087e-04,
            2.2527e-04,  2.4128e-04],
          [ 5.6434e-04,  5.6699e-04,  5.7902e-04,  ...,  1.9537e-04,
            2.1069e-04,  2.2783e-04],
          [ 5.5708e-04,  5.5848e-04,  5.6905e-04,  ...,  1.8418e-04,
            1.9989e-04,  2.1764e-04],
          ...,
          [ 7.8438e-04,  8.4481e-04,  9.0726e-04,  ...,  2.2228e-03,
            2.1675e-03,  2.0991e-03],
          [ 7.4743e-04,  8.0429e-04,  8.6307e-04,  ...,  2.1938e-03,
            2.1427e-03,  2.0781e-03],
          [ 7.0935e-04,  7.6231e-04,  8.1705e-04,  ...,  2.1476e-03,
            2.1008e-03,  2.0405e-03]]],


        [[[ 1.1054e-04,  1.2076e-04,  1.3515e-04,  ...,  3.2011e-05,
            4.0286e-05,  4.9567e-05],
          [ 1.0851e-04,  1.1894e-04,  1.3353e-04,  ...,  2.3166e-05,
            3.1917e-05,  4.1793e-05],
          [ 1.0811e-04,  1.1839e-04,  1.3268e-04,  ...,  1.6555e-05,
            2.5523e-05,  3.5739e-05],
          ...,
          [ 5.9358e-05,  6.1944e-05,  6.5812e-05,  ...,  5.2887e-04,
            5.1426e-04,  4.9638e-04],
          [ 5.9398e-05,  6.1865e-05,  6.5522e-05,  ...,  5.2699e-04,
            5.1330e-04,  4.9614e-04],
          [ 5.9998e-05,  6.2366e-05,  6.5825e-05,  ...,  5.1971e-04,
            5.0699e-04,  4.9075e-04]]],


        [[[-7.2305e-05, -6.8891e-05, -6.5782e-05,  ..., -4.6045e-05,
           -4.0463e-05, -3.3982e-05],
          [-7.1842e-05, -6.8266e-05, -6.5024e-05,  ..., -5.2727e-05,
           -4.6774e-05, -3.9835e-05],
          [-7.2219e-05, -6.8596e-05, -6.5331e-05,  ..., -5.7980e-05,
           -5.1791e-05, -4.4535e-05],
          ...,
          [-4.1959e-05, -4.7802e-05, -5.3610e-05,  ..., -1.4675e-04,
           -1.4492e-04, -1.4238e-04],
          [-3.7563e-05, -4.3023e-05, -4.8488e-05,  ..., -1.4271e-04,
           -1.4109e-04, -1.3882e-04],
          [-3.3090e-05, -3.8117e-05, -4.3175e-05,  ..., -1.3850e-04,
           -1.3709e-04, -1.3510e-04]]],


        [[[-8.0156e-05, -8.0910e-05, -8.2155e-05,  ..., -1.9987e-05,
           -2.2388e-05, -2.5104e-05],
          [-7.9043e-05, -7.9730e-05, -8.0924e-05,  ..., -1.7320e-05,
           -1.9882e-05, -2.2794e-05],
          [-7.8055e-05, -7.8650e-05, -7.9751e-05,  ..., -1.5367e-05,
           -1.8008e-05, -2.1036e-05],
          ...,
          [-3.3166e-05, -3.9366e-05, -4.6103e-05,  ..., -1.5049e-04,
           -1.4525e-04, -1.3942e-04],
          [-2.9537e-05, -3.5346e-05, -4.1672e-05,  ..., -1.4870e-04,
           -1.4385e-04, -1.3837e-04],
          [-2.5997e-05, -3.1384e-05, -3.7255e-05,  ..., -1.4571e-04,
           -1.4128e-04, -1.3618e-04]]],


        [[[-5.4506e-05, -5.9346e-05, -6.5622e-05,  ..., -3.6163e-05,
           -4.2495e-05, -4.9580e-05],
          [-5.4123e-05, -5.9050e-05, -6.5398e-05,  ..., -2.9271e-05,
           -3.5993e-05, -4.3567e-05],
          [-5.3993e-05, -5.8813e-05, -6.4969e-05,  ..., -2.3810e-05,
           -3.0751e-05, -3.8649e-05],
          ...,
          [-1.8005e-04, -1.8757e-04, -1.9524e-04,  ..., -1.3448e-05,
           -9.3281e-06, -5.7736e-06],
          [-1.7425e-04, -1.8118e-04, -1.8821e-04,  ..., -1.6242e-05,
           -1.2281e-05, -8.7322e-06],
          [-1.6839e-04, -1.7475e-04, -1.8118e-04,  ..., -1.7972e-05,
           -1.4214e-05, -1.0738e-05]]]])
In [74]:
In [75]:
In [76]:
In [77]:
Max: 6.421447277069092, Min: -6.50855827331543, Avg: 3.442086017457768e-05, Std:0.040053438395261765
torch.Size([5, 32, 160, 160])
Out[77]:
(tensor(6.4214), tensor(-6.5086), tensor(3.4421e-05), tensor(0.0401))
In [78]:
torch.Size([5, 1, 160, 160])
In [79]:
Max: 0.06074824929237366, Min: -0.15873178839683533, Avg: -2.053680191238527e-06, Std:0.0036773306783288717
torch.Size([5, 1, 640, 640])
Out[79]:
tensor([[[[ 6.8180e-04,  7.0503e-04,  7.3616e-04,  ...,  2.3364e-04,
            2.4231e-04,  2.5344e-04],
          [ 6.8369e-04,  7.0946e-04,  7.4339e-04,  ...,  2.3728e-04,
            2.4405e-04,  2.5340e-04],
          [ 6.8980e-04,  7.1771e-04,  7.5379e-04,  ...,  2.5019e-04,
            2.5401e-04,  2.6047e-04],
          ...,
          [ 1.9389e-03,  2.1803e-03,  2.3906e-03,  ...,  3.4239e-03,
            3.3213e-03,  3.1524e-03],
          [ 1.8414e-03,  2.0697e-03,  2.2688e-03,  ...,  3.3222e-03,
            3.2270e-03,  3.0682e-03],
          [ 1.6975e-03,  1.9055e-03,  2.0872e-03,  ...,  3.1593e-03,
            3.0732e-03,  2.9287e-03]]],


        [[[ 1.6637e-04,  1.8620e-04,  2.0912e-04,  ...,  4.9994e-05,
            5.6328e-05,  6.3331e-05],
          [ 1.7505e-04,  1.9746e-04,  2.2298e-04,  ...,  4.9748e-05,
            5.5466e-05,  6.1876e-05],
          [ 1.8566e-04,  2.1056e-04,  2.3847e-04,  ...,  5.3371e-05,
            5.7989e-05,  6.3316e-05],
          ...,
          [ 6.1035e-04,  7.0100e-04,  7.7825e-04,  ...,  9.3303e-04,
            9.0003e-04,  8.4670e-04],
          [ 5.8327e-04,  6.6941e-04,  7.4272e-04,  ...,  9.0024e-04,
            8.7016e-04,  8.2067e-04],
          [ 5.3455e-04,  6.1270e-04,  6.7922e-04,  ...,  8.4956e-04,
            8.2284e-04,  7.7836e-04]]],


        [[[-9.4671e-05, -9.3465e-05, -9.1411e-05,  ..., -3.1525e-05,
           -2.6396e-05, -2.1095e-05],
          [-9.5004e-05, -9.3507e-05, -9.1076e-05,  ..., -3.3117e-05,
           -2.8141e-05, -2.3013e-05],
          [-9.4413e-05, -9.2470e-05, -8.9559e-05,  ..., -3.2682e-05,
           -2.8136e-05, -2.3443e-05],
          ...,
          [ 6.9573e-05,  8.3259e-05,  9.4769e-05,  ..., -1.6244e-04,
           -1.5905e-04, -1.5460e-04],
          [ 6.7672e-05,  8.0713e-05,  9.1678e-05,  ..., -1.6350e-04,
           -1.5951e-04, -1.5453e-04],
          [ 6.1678e-05,  7.3375e-05,  8.3193e-05,  ..., -1.6165e-04,
           -1.5745e-04, -1.5235e-04]]],


        [[[-5.2182e-05, -5.1073e-05, -5.2229e-05,  ..., -2.5946e-05,
           -2.7734e-05, -2.9718e-05],
          [-4.7158e-05, -4.5697e-05, -4.6766e-05,  ..., -2.5913e-05,
           -2.7520e-05, -2.9333e-05],
          [-4.3330e-05, -4.1570e-05, -4.2526e-05,  ..., -2.7056e-05,
           -2.8331e-05, -2.9822e-05],
          ...,
          [-2.0790e-04, -2.4217e-04, -2.7223e-04,  ..., -2.6880e-04,
           -2.6025e-04, -2.4533e-04],
          [-1.9566e-04, -2.2811e-04, -2.5653e-04,  ..., -2.6038e-04,
           -2.5254e-04, -2.3857e-04],
          [-1.7645e-04, -2.0593e-04, -2.3178e-04,  ..., -2.4618e-04,
           -2.3917e-04, -2.2651e-04]]],


        [[[-1.0228e-04, -1.1267e-04, -1.2248e-04,  ..., -5.5168e-05,
           -6.0544e-05, -6.5904e-05],
          [-1.1107e-04, -1.2296e-04, -1.3399e-04,  ..., -5.7635e-05,
           -6.2433e-05, -6.7124e-05],
          [-1.1990e-04, -1.3320e-04, -1.4536e-04,  ..., -6.3339e-05,
           -6.7129e-05, -7.0714e-05],
          ...,
          [-5.3089e-04, -5.9228e-04, -6.4342e-04,  ..., -1.3773e-04,
           -1.3156e-04, -1.1932e-04],
          [-5.0607e-04, -5.6408e-04, -6.1240e-04,  ..., -1.2532e-04,
           -1.2083e-04, -1.1045e-04],
          [-4.6769e-04, -5.2024e-04, -5.6407e-04,  ..., -1.1002e-04,
           -1.0682e-04, -9.8210e-05]]]])
In [80]:
In [81]:
In [82]:
In [83]:
Max: 6.783955097198486, Min: -6.637836456298828, Avg: 8.644248623568274e-07, Std:0.01757976971566677
torch.Size([5, 32, 320, 320])
Out[83]:
(tensor(6.7840), tensor(-6.6378), tensor(8.6442e-07), tensor(0.0176))
In [84]:
torch.Size([5, 1, 320, 320])
In [85]:
Max: 0.5269816517829895, Min: -0.5193753838539124, Avg: 6.192673754412681e-05, Std:0.012026630342006683
torch.Size([5, 1, 640, 640])
Out[85]:
tensor([[[[ 4.6049e-04,  5.4600e-04,  6.6998e-04,  ...,  5.6103e-04,
            5.0241e-04,  4.1854e-04],
          [ 4.2309e-04,  5.3103e-04,  6.8867e-04,  ...,  7.0519e-04,
            6.1907e-04,  4.9897e-04],
          [ 4.1165e-04,  5.3549e-04,  7.1819e-04,  ...,  8.5342e-04,
            7.4031e-04,  5.8624e-04],
          ...,
          [ 3.8127e-03,  4.6678e-03,  5.3933e-03,  ...,  5.3806e-03,
            5.8471e-03,  5.8297e-03],
          [ 3.3803e-03,  4.1185e-03,  4.7514e-03,  ...,  5.0744e-03,
            5.5322e-03,  5.5394e-03],
          [ 2.8174e-03,  3.4151e-03,  3.9384e-03,  ...,  4.5503e-03,
            4.9460e-03,  4.9638e-03]]],


        [[[ 2.6523e-04,  3.3110e-04,  3.8704e-04,  ...,  2.2587e-04,
            2.0956e-04,  1.7625e-04],
          [ 2.9064e-04,  3.7472e-04,  4.5084e-04,  ...,  2.9232e-04,
            2.6624e-04,  2.1811e-04],
          [ 3.1130e-04,  4.1222e-04,  5.0940e-04,  ...,  3.5589e-04,
            3.1993e-04,  2.5821e-04],
          ...,
          [ 1.6120e-03,  1.9949e-03,  2.3035e-03,  ...,  1.7152e-03,
            1.8621e-03,  1.8368e-03],
          [ 1.3846e-03,  1.7072e-03,  1.9706e-03,  ...,  1.5968e-03,
            1.7440e-03,  1.7312e-03],
          [ 1.1024e-03,  1.3554e-03,  1.5673e-03,  ...,  1.4000e-03,
            1.5301e-03,  1.5263e-03]]],


        [[[-3.6078e-05, -3.8865e-05, -4.8814e-05,  ...,  2.4038e-05,
            2.3325e-05,  1.6203e-05],
          [-2.2111e-05, -2.3954e-05, -3.4232e-05,  ...,  3.9766e-05,
            3.7383e-05,  2.6741e-05],
          [-9.0334e-06, -7.1720e-06, -1.4315e-05,  ...,  5.4485e-05,
            5.0677e-05,  3.7162e-05],
          ...,
          [ 3.1002e-04,  3.9032e-04,  4.5415e-04,  ..., -1.1953e-04,
           -1.1556e-04, -1.1730e-04],
          [ 2.5055e-04,  3.1497e-04,  3.6703e-04,  ..., -1.1745e-04,
           -1.1234e-04, -1.1322e-04],
          [ 1.8378e-04,  2.3120e-04,  2.7062e-04,  ..., -1.2115e-04,
           -1.1476e-04, -1.1371e-04]]],


        [[[ 5.6377e-05,  7.2268e-05,  7.1998e-05,  ..., -7.7748e-05,
           -7.2084e-05, -6.1587e-05],
          [ 8.1780e-05,  1.0217e-04,  1.0378e-04,  ..., -9.8054e-05,
           -8.9267e-05, -7.4142e-05],
          [ 9.4312e-05,  1.1826e-04,  1.2258e-04,  ..., -1.1779e-04,
           -1.0587e-04, -8.6485e-05],
          ...,
          [-5.3017e-04, -6.6631e-04, -7.8110e-04,  ..., -5.7232e-04,
           -6.1272e-04, -5.9587e-04],
          [-4.5916e-04, -5.7622e-04, -6.7627e-04,  ..., -5.2976e-04,
           -5.7331e-04, -5.6278e-04],
          [-3.6718e-04, -4.6120e-04, -5.4350e-04,  ..., -4.5988e-04,
           -5.0026e-04, -4.9475e-04]]],


        [[[-2.5030e-04, -2.9747e-04, -3.2570e-04,  ..., -1.9521e-04,
           -1.7851e-04, -1.4846e-04],
          [-2.9220e-04, -3.5383e-04, -3.9463e-04,  ..., -2.5239e-04,
           -2.2681e-04, -1.8375e-04],
          [-3.2217e-04, -3.9775e-04, -4.5321e-04,  ..., -3.0804e-04,
           -2.7384e-04, -2.1894e-04],
          ...,
          [-1.2255e-03, -1.4871e-03, -1.6969e-03,  ..., -5.8839e-04,
           -6.2905e-04, -5.9456e-04],
          [-1.0648e-03, -1.2843e-03, -1.4619e-03,  ..., -5.3237e-04,
           -5.7980e-04, -5.5578e-04],
          [-8.6732e-04, -1.0389e-03, -1.1807e-03,  ..., -4.3586e-04,
           -4.8369e-04, -4.7074e-04]]]])
In [86]:
In [87]:
In [88]:
In [89]:
Max: 18.470808029174805, Min: -16.491992950439453, Avg: -1.3220964319771156e-05, Std:0.0798465833067894
torch.Size([5, 16, 320, 320])
Out[89]:
(tensor(18.4708), tensor(-16.4920), tensor(-1.3221e-05), tensor(0.0798))
In [90]:
torch.Size([5, 1, 320, 320])
In [91]:
Max: 0.8531064391136169, Min: -0.9852874279022217, Avg: 5.979713023407385e-05, Std:0.019329551607370377
torch.Size([5, 1, 640, 640])
Out[91]:
tensor([[[[ 2.8614e-04,  3.7336e-04,  5.5900e-04,  ...,  1.3668e-03,
            1.1937e-03,  9.1158e-04],
          [ 2.1959e-04,  4.0791e-04,  7.4349e-04,  ...,  1.8154e-03,
            1.5416e-03,  1.1358e-03],
          [ 2.2994e-04,  5.4193e-04,  1.0410e-03,  ...,  2.1244e-03,
            1.7601e-03,  1.2640e-03],
          ...,
          [ 8.6956e-03,  1.1755e-02,  1.3941e-02,  ...,  1.5756e-02,
            1.6436e-02,  1.4535e-02],
          [ 7.4541e-03,  1.0049e-02,  1.1925e-02,  ...,  1.5794e-02,
            1.5531e-02,  1.3210e-02],
          [ 5.6418e-03,  7.5378e-03,  8.9413e-03,  ...,  1.3444e-02,
            1.2700e-02,  1.0564e-02]]],


        [[[ 5.6974e-04,  7.5642e-04,  8.8177e-04,  ...,  7.0133e-04,
            6.3185e-04,  4.8741e-04],
          [ 7.2503e-04,  1.0103e-03,  1.2239e-03,  ...,  9.2320e-04,
            8.1686e-04,  6.1721e-04],
          [ 8.2724e-04,  1.2006e-03,  1.5056e-03,  ...,  1.0485e-03,
            9.1229e-04,  6.7911e-04],
          ...,
          [ 4.4459e-03,  6.0383e-03,  7.0955e-03,  ...,  5.6730e-03,
            5.8245e-03,  5.0500e-03],
          [ 3.7758e-03,  5.0730e-03,  5.9075e-03,  ...,  5.5896e-03,
            5.4213e-03,  4.5220e-03],
          [ 2.7791e-03,  3.6827e-03,  4.2517e-03,  ...,  4.6571e-03,
            4.3404e-03,  3.5362e-03]]],


        [[[ 3.4213e-05,  5.0339e-05,  4.3600e-05,  ...,  2.4677e-04,
            2.2236e-04,  1.6359e-04],
          [ 9.2251e-05,  1.2406e-04,  1.2292e-04,  ...,  3.0646e-04,
            2.7742e-04,  2.0562e-04],
          [ 1.4227e-04,  1.9148e-04,  2.0027e-04,  ...,  3.1081e-04,
            2.8377e-04,  2.1241e-04],
          ...,
          [ 9.7952e-04,  1.3643e-03,  1.6296e-03,  ...,  1.1835e-04,
            1.1964e-04,  7.1876e-05],
          [ 8.0067e-04,  1.0941e-03,  1.2831e-03,  ...,  1.0538e-04,
            8.2717e-05,  2.8319e-05],
          [ 5.5894e-04,  7.4704e-04,  8.5910e-04,  ...,  4.9989e-05,
            1.8940e-05, -2.7740e-05]]],


        [[[ 2.9187e-04,  3.8565e-04,  4.1638e-04,  ..., -2.2041e-04,
           -1.9725e-04, -1.5278e-04],
          [ 3.8093e-04,  4.9268e-04,  5.2348e-04,  ..., -2.8645e-04,
           -2.5152e-04, -1.9025e-04],
          [ 4.0441e-04,  5.1328e-04,  5.3473e-04,  ..., -3.2350e-04,
           -2.7919e-04, -2.0781e-04],
          ...,
          [-1.3984e-03, -1.9195e-03, -2.2837e-03,  ..., -2.0004e-03,
           -1.9940e-03, -1.6874e-03],
          [-1.1846e-03, -1.6202e-03, -1.9248e-03,  ..., -1.9611e-03,
           -1.8613e-03, -1.5210e-03],
          [-8.7089e-04, -1.1836e-03, -1.4052e-03,  ..., -1.6220e-03,
           -1.4886e-03, -1.1920e-03]]],


        [[[-6.2019e-04, -8.0666e-04, -9.0759e-04,  ..., -5.7526e-04,
           -5.1206e-04, -3.9158e-04],
          [-7.8577e-04, -1.0441e-03, -1.1963e-03,  ..., -7.4995e-04,
           -6.5610e-04, -4.9153e-04],
          [-8.6613e-04, -1.1722e-03, -1.3689e-03,  ..., -8.4578e-04,
           -7.2839e-04, -5.3822e-04],
          ...,
          [-3.0768e-03, -4.1764e-03, -4.9400e-03,  ..., -2.6306e-03,
           -2.6080e-03, -2.1611e-03],
          [-2.6133e-03, -3.4995e-03, -4.0950e-03,  ..., -2.5293e-03,
           -2.3731e-03, -1.8877e-03],
          [-1.9432e-03, -2.5535e-03, -2.9529e-03,  ..., -2.0271e-03,
           -1.8260e-03, -1.4099e-03]]]])
In [92]:
In [93]:
In [94]:
In [95]:
In [96]:
Max: 19.469928741455078, Min: -15.725606918334961, Avg: 9.911394727168954e-07, Std:0.03895791247487068
torch.Size([5, 16, 640, 640])
Out[96]:
(tensor(19.4699), tensor(-15.7256), tensor(9.9114e-07), tensor(0.0390))
In [97]:
torch.Size([5, 1, 640, 640])
In [98]:
Max: 10.160104751586914, Min: -7.765002250671387, Avg: -6.873896927572787e-05, Std:0.10834283381700516
torch.Size([5, 1, 640, 640])
Out[98]:
tensor([[[[ 2.5875e-03, -1.0001e-03, -1.1211e-03,  ...,  1.2672e-03,
            9.6760e-04,  8.9497e-04],
          [-1.1697e-03, -4.1260e-03, -3.3476e-03,  ...,  1.9068e-03,
            1.9689e-03,  1.5664e-03],
          [-2.3113e-03, -4.5324e-03, -3.2343e-03,  ...,  1.9833e-03,
            2.2320e-03,  1.7983e-03],
          ...,
          [ 1.2939e-02,  2.0172e-02,  2.0174e-02,  ...,  1.6620e-02,
            2.0674e-02,  1.7868e-02],
          [ 1.2229e-02,  2.0359e-02,  2.0127e-02,  ...,  2.0197e-02,
            3.0809e-02,  2.7618e-02],
          [ 8.6463e-03,  1.3626e-02,  1.3956e-02,  ...,  1.5852e-02,
            2.5617e-02,  2.3821e-02]]],


        [[[ 8.2520e-04,  1.2229e-04,  9.8726e-05,  ...,  8.1094e-04,
            7.2495e-04,  5.9219e-04],
          [-5.0220e-05, -3.8561e-04, -2.5133e-04,  ...,  1.2260e-03,
            1.3773e-03,  1.0755e-03],
          [-3.6796e-04, -6.9085e-04, -3.2079e-04,  ...,  1.2156e-03,
            1.4254e-03,  1.1412e-03],
          ...,
          [ 6.3284e-03,  1.0784e-02,  1.0697e-02,  ...,  6.0408e-03,
            7.3743e-03,  6.2667e-03],
          [ 6.7626e-03,  1.2269e-02,  1.1680e-02,  ...,  7.2973e-03,
            1.1043e-02,  9.8348e-03],
          [ 5.0910e-03,  8.7737e-03,  8.5171e-03,  ...,  5.6102e-03,
            9.1053e-03,  8.4340e-03]]],


        [[[-1.7831e-04, -1.6540e-04, -2.2773e-04,  ...,  3.6071e-04,
            3.6085e-04,  2.7487e-04],
          [-2.1501e-04, -2.4600e-04, -3.9392e-04,  ...,  5.8762e-04,
            7.2618e-04,  5.6312e-04],
          [-1.5259e-04, -2.4994e-04, -4.1274e-04,  ...,  5.4930e-04,
            6.9642e-04,  5.4729e-04],
          ...,
          [ 1.3973e-03,  2.5537e-03,  2.5686e-03,  ...,  5.2613e-05,
            1.4881e-04,  1.3000e-04],
          [ 1.5813e-03,  3.0895e-03,  2.9215e-03,  ...,  2.0251e-04,
            4.2587e-04,  3.8474e-04],
          [ 1.2484e-03,  2.2999e-03,  2.1930e-03,  ...,  1.2361e-04,
            3.3211e-04,  3.0492e-04]]],


        [[[-3.7276e-04,  7.7541e-04,  9.0643e-04,  ..., -2.4137e-04,
           -2.1276e-04, -1.7972e-04],
          [ 8.6678e-04,  1.9964e-03,  1.8647e-03,  ..., -3.7183e-04,
           -4.1408e-04, -3.2487e-04],
          [ 1.1533e-03,  1.9644e-03,  1.7845e-03,  ..., -3.7010e-04,
           -4.3062e-04, -3.4306e-04],
          ...,
          [-2.1253e-03, -3.4116e-03, -3.3859e-03,  ..., -2.1687e-03,
           -2.5838e-03, -2.1738e-03],
          [-2.1156e-03, -3.6585e-03, -3.5626e-03,  ..., -2.6206e-03,
           -3.8786e-03, -3.4437e-03],
          [-1.4986e-03, -2.5021e-03, -2.5178e-03,  ..., -2.0323e-03,
           -3.2104e-03, -2.9592e-03]]],


        [[[-2.6828e-05, -7.8177e-04, -9.0311e-04,  ..., -6.6193e-04,
           -5.7724e-04, -4.7860e-04],
          [-8.9286e-04, -1.7133e-03, -1.5739e-03,  ..., -1.0225e-03,
           -1.1364e-03, -8.8565e-04],
          [-1.1086e-03, -1.5616e-03, -1.4670e-03,  ..., -1.0221e-03,
           -1.1812e-03, -9.2916e-04],
          ...,
          [-4.4586e-03, -7.5040e-03, -7.4616e-03,  ..., -2.7234e-03,
           -3.2976e-03, -2.7875e-03],
          [-4.8164e-03, -8.6702e-03, -8.2601e-03,  ..., -3.4004e-03,
           -5.1519e-03, -4.6006e-03],
          [-3.7036e-03, -6.3008e-03, -6.1029e-03,  ..., -2.5862e-03,
           -4.2422e-03, -3.9213e-03]]]])
In [99]:
In [100]:
In [101]:
In [102]:
In [103]:
Max: 20.467784881591797, Min: -25.98050308227539, Avg: -2.3751272237859666e-06, Std:0.1460483819246292
torch.Size([5, 3, 640, 640])
In [104]:
torch.Size([5, 1, 640, 640])
In [105]:
Max: 14.537252426147461, Min: -10.553370475769043, Avg: 2.2986019757809117e-05, Std:0.1260661482810974
torch.Size([5, 1, 640, 640])
Out[105]:
tensor([[[[ 3.4371e-03,  5.9441e-04,  3.1330e-03,  ...,  1.6768e-03,
            1.2496e-03,  1.0991e-03],
          [ 1.0900e-03, -1.6565e-03, -1.0456e-03,  ...,  2.0946e-03,
            2.2932e-03,  1.8843e-03],
          [ 5.6859e-03,  2.3292e-03, -5.3299e-04,  ...,  2.8936e-03,
            2.6448e-03,  2.1848e-03],
          ...,
          [ 9.2655e-03,  2.0208e-02,  2.3482e-02,  ...,  1.3104e-02,
            1.5440e-02,  1.6472e-02],
          [ 8.9934e-03,  1.8358e-02,  1.9338e-02,  ...,  1.6909e-02,
            2.6366e-02,  2.6378e-02],
          [ 8.5512e-03,  1.3883e-02,  1.4292e-02,  ...,  1.6093e-02,
            2.6140e-02,  2.4301e-02]]],


        [[[ 3.2269e-03,  2.1824e-03,  2.7453e-03,  ...,  9.2099e-04,
            6.8989e-04,  6.1254e-04],
          [ 3.5140e-03,  2.2835e-03,  2.0108e-03,  ...,  1.2012e-03,
            1.3478e-03,  1.1547e-03],
          [ 4.5961e-03,  3.2475e-03,  2.2859e-03,  ...,  1.4341e-03,
            1.4033e-03,  1.2304e-03],
          ...,
          [ 3.9697e-03,  9.6253e-03,  1.0775e-02,  ...,  4.6622e-03,
            5.3416e-03,  5.6972e-03],
          [ 4.5604e-03,  1.0446e-02,  1.0463e-02,  ...,  6.0283e-03,
            9.3262e-03,  9.3314e-03],
          [ 4.9641e-03,  8.9559e-03,  8.7254e-03,  ...,  5.6986e-03,
            9.2919e-03,  8.6035e-03]]],


        [[[ 7.0222e-04,  7.0121e-04,  7.8724e-04,  ...,  4.1217e-04,
            2.7234e-04,  2.4702e-04],
          [ 1.0533e-03,  8.6393e-04,  8.5478e-04,  ...,  4.7854e-04,
            6.0869e-04,  5.6247e-04],
          [ 7.5107e-04,  5.6856e-04,  8.1140e-04,  ...,  3.9907e-04,
            4.9707e-04,  5.0687e-04],
          ...,
          [ 5.0342e-04,  1.8586e-03,  2.0410e-03,  ..., -1.4441e-04,
           -4.9085e-05,  9.4963e-05],
          [ 8.3777e-04,  2.4440e-03,  2.4432e-03,  ...,  1.3504e-04,
            3.0251e-04,  3.4628e-04],
          [ 1.1877e-03,  2.3475e-03,  2.2541e-03,  ...,  1.3678e-04,
            3.4573e-04,  3.1477e-04]]],


        [[[ 3.0578e-04,  8.6114e-04,  1.6962e-04,  ..., -2.9225e-04,
           -2.0932e-04, -1.8917e-04],
          [ 1.2576e-03,  1.7539e-03,  1.3601e-03,  ..., -3.6936e-04,
           -4.1010e-04, -3.5277e-04],
          [ 4.4010e-04,  1.0628e-03,  1.4583e-03,  ..., -4.3778e-04,
           -4.2517e-04, -3.7135e-04],
          ...,
          [-1.3822e-03, -3.2218e-03, -3.6931e-03,  ..., -1.8228e-03,
           -1.9866e-03, -2.0313e-03],
          [-1.4612e-03, -3.1956e-03, -3.3072e-03,  ..., -2.1510e-03,
           -3.3290e-03, -3.3062e-03],
          [-1.4705e-03, -2.5531e-03, -2.5828e-03,  ..., -2.0707e-03,
           -3.2912e-03, -3.0323e-03]]],


        [[[-2.2239e-03, -2.2903e-03, -2.1074e-03,  ..., -7.9820e-04,
           -5.8675e-04, -5.1517e-04],
          [-3.3577e-03, -3.1318e-03, -2.7511e-03,  ..., -9.6319e-04,
           -1.1060e-03, -9.5593e-04],
          [-3.1221e-03, -2.8956e-03, -3.0032e-03,  ..., -1.1212e-03,
           -1.1087e-03, -9.7954e-04],
          ...,
          [-2.2693e-03, -6.2072e-03, -7.0217e-03,  ..., -2.2686e-03,
           -2.4738e-03, -2.6508e-03],
          [-2.9337e-03, -7.2321e-03, -7.3113e-03,  ..., -2.8467e-03,
           -4.4117e-03, -4.4208e-03],
          [-3.5358e-03, -6.4263e-03, -6.2744e-03,  ..., -2.6588e-03,
           -4.3636e-03, -4.0243e-03]]]])

We can decide to sum score_accumulated_k or not!!

In [106]:
In [107]:
Part of the score represented by last layer:
tensor([  2.1240, -12.2649,   0.3176,  -6.5317,   1.7537])
tensor([-452.4433, -216.7106,  -53.5572,   44.1896,   71.3443])
tensor([-450.3685, -228.4828,  -53.1888,   37.3385,   73.2811],
       grad_fn=<AddBackward0>)
In [108]:
tensor([[  2.1239, -12.2650,   0.3176,  -6.5317,   1.7537]])
In [109]:
tensor([[-450.3685, -228.4829,  -53.1888,   37.3385,   73.2811]],
       grad_fn=<AddBackward0>)
torch.Size([1, 5, 640, 640])
In [112]:
Class 0 : -450.3677978515625 == -450.36859130859375 
Class 1 : -228.48361206054688 == -228.48291015625 
Class 2 : -53.18899154663086 == -53.188819885253906 
Class 3 : 37.33872604370117 == 37.33848190307617 
Class 4 : 73.27993774414062 == 73.28113555908203 
Score 637
tensor([[ 352.9369,  130.4362,   23.8573,  -27.6692, -127.2996]])
Score 509
tensor([[-238.5746, -136.5996,  -82.1229,   27.9251,   95.6821]])
Score 381
tensor([[ 90.3425,  -4.0964, -71.5431,  17.6218,  56.2163]])
Score 253
tensor([[ 69.3464,  90.1305,  51.9393, -24.0574, -94.7809]])
Score 189
tensor([[ -67.4019,  -80.3981, -105.9042,  -30.5506,  131.9613]])
Score 125
tensor([[-268.0844, -197.0137,  -35.3323,   19.2853,  130.5606]])
Score 93
tensor([[-435.0080,  -98.9849,   41.1478,   88.3761,   -2.8188]])
Score 61
tensor([[-432.6903, -125.3472,   51.2873,   41.9467,   35.8731]])
Score 45
tensor([[320.8301,  66.8347, -21.3885, -40.5836, -23.8067]])
Score 29
tensor([[ 81.3512,  36.8573,  30.9944, -17.8175, -52.4265]])
Score 21
tensor([[-24.9841,  19.3752,  21.8447,   1.1939, -21.6358]])
Score 13
tensor([[137.5521,  26.9112, -22.6982, -19.4071,   4.4679]])
Score 9
tensor([[ 39.4282,  64.8473,  38.3560,  29.1912, -49.3582]])
Score 5
tensor([[-131.3264,  -28.6922,   30.6692,  -10.9151,   -0.5127]])
Score 3
tensor([[ 53.8388,  19.0292,  -4.6643, -10.3499, -10.7776]])
In [114]:
Score distribution for predicted class: 4
In [115]:
In [ ]:

A intermediate map visualization

In [116]:
In [117]:
In [118]:
Score k for class 4:
max=2.82635498046875
min=-3.720142364501953
sum=71.34439086914062
std=0.06187132000923157
mean=0.00017418064817320555
ne0=0%
In [119]:
Score input for class 4:
max=19.601152420043945
min=-9.380825996398926
sum=1.7538375854492188
std=0.17728756368160248
mean=4.281830115360208e-06
ne0=0%
In [120]:
Score total for class 4:
max=20.171215057373047
min=-9.948592185974121
sum=73.09830474853516
std=0.19448599219322205
mean=0.0001784626510925591
ne0=0%
In [ ]:
In [ ]: